Exploring Target Representations for Masked Autoencoders

Masked autoencoders have become popular training paradigms for self-supervised visual representation learning. These models randomly mask a portion of the input and reconstruct the masked portion according to the target representations. In this paper, we first show that a careful choice of the target representation is unnecessary for learning good representations, since different targets tend to derive similarly behaved models. Driven by this observation, we propose a multi-stage masked distillation pipeline and use a randomly initialized model as the teacher, enabling us to effectively train high-capacity models without any efforts to carefully design target representations. Interestingly, we further explore using teachers of larger capacity, obtaining distilled students with remarkable transferring ability. On different tasks of classification, transfer learning, object detection, and semantic segmentation, the proposed method to perform masked knowledge distillation with bootstrapped teachers (dBOT) outperforms previous self-supervised methods by nontrivial margins. We hope our findings, as well as the proposed method, could motivate people to rethink the roles of target representations in pre-training masked autoencoders.The code and pre-trained models are publicly available at https://github.com/liuxingbin/dbot.Comment: The first two authors contributed equall

Latent Feature Relation Consistency for Adversarial Robustness

Deep neural networks have been applied in many computer vision tasks and achieved state-of-the-art performance. However, misclassification will occur when DNN predicts adversarial examples which add human-imperceptible adversarial noise to natural examples. This limits the application of DNN in security-critical fields. To alleviate this problem, we first conducted an empirical analysis of the latent features of both adversarial and natural examples and found the similarity matrix of natural examples is more compact than those of adversarial examples. Motivated by this observation, we propose \textbf{L}atent \textbf{F}eature \textbf{R}elation \textbf{C}onsistency (\textbf{LFRC}), which constrains the relation of adversarial examples in latent space to be consistent with the natural examples. Importantly, our LFRC is orthogonal to the previous method and can be easily combined with them to achieve further improvement. To demonstrate the effectiveness of LFRC, we conduct extensive experiments using different neural networks on benchmark datasets. For instance, LFRC can bring 0.78\% further improvement compared to AT, and 1.09\% improvement compared to TRADES, against AutoAttack on CIFAR10. Code is available at https://github.com/liuxingbin/LFRC.Comment: Tech repor

CAT:Collaborative Adversarial Training

Adversarial training can improve the robustness of neural networks. Previous methods focus on a single adversarial training strategy and do not consider the model property trained by different strategies. By revisiting the previous methods, we find different adversarial training methods have distinct robustness for sample instances. For example, a sample instance can be correctly classified by a model trained using standard adversarial training (AT) but not by a model trained using TRADES, and vice versa. Based on this observation, we propose a collaborative adversarial training framework to improve the robustness of neural networks. Specifically, we use different adversarial training methods to train robust models and let models interact with their knowledge during the training process. Collaborative Adversarial Training (CAT) can improve both robustness and accuracy. Extensive experiments on various networks and datasets validate the effectiveness of our method. CAT achieves state-of-the-art adversarial robustness without using any additional data on CIFAR-10 under the Auto-Attack benchmark. Code is available at https://github.com/liuxingbin/CAT.Comment: Tech repor

Simultaneous separation and purification of chlorogenic acid, epicatechin, hyperoside and phlorizin from thinned young Qinguan apples by successive use of polyethylene and polyamide resins

The method for separating and purifying chlorogenic acid (CA), epicatechin (EC), hyperoside (HY) and phlorizin (PH) simutaneously from young Qinguan apples by successive use of X-5 and polyamide resins has been developed in this study. The order of adsorption capacities of X-5 for the four phenolics was PH\ua0>\ua0HY\ua0>\ua0EC\ua0>\ua0CA, and the adsorption equilibriums of the four phenolics onto X-5 resin conformed to Langmuir isotherms preferentially. The adsorption kinetics of EC and CA onto X-5 conformed to the pseudo-first-order model, while that of HY and PH accorded with the pseudo-second-order model. Interestingly, the values of equilibrium adsorption capacities (Q) calculated in the preferential kinetics models were closer to that of theoretical maximum adsorption capacities (Q) calculated by Langmuir isotherms. Through dynamic adsorption and desorption using X-5 and polyamide resins with ethanol solution as strippant, CA, EC, HY and PH were obtained with purities of 96.21%, 95.34%, 95.36% and 97.36%, respectively

Analysis between ABO blood group and clinical outcomes in COVID-19 patients and the potential mediating role of ACE2

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has become the most common coronavirus that causes large-scale infections worldwide. Currently, several studies have shown that the ABO blood group is associated with coronavirus disease 2019 (COVID-19) infection and some studies have also suggested that the infection of COVID-19 may be closely related to the interaction between angiotensin-converting enzyme 2 (ACE2) and blood group antigens. However, the relationship between blood type to clinical outcome in critically ill patients and the mechanism of action is still unclear. The current study aimed to examine the correlation between blood type distribution and SARS-CoV-2 infection, progression, and prognosis in patients with COVID-19 and the potential mediating role of ACE2. With 234 patients from 5 medical centers and two established cohorts, 137 for the mild cohort and 97 for the critically ill cohort, we found that the blood type A population was more sensitive to SARS-CoV-2, while the blood type distribution was not relevant to acute respiratory distress syndrome (ARDS), acute kidney injury (AKI), and mortality in COVID-19 patients. Further study showed that the serum ACE2 protein level of healthy people with type A was significantly higher than that of other blood groups, and type O was the lowest. The experimental results of spike protein binding to red blood cells also showed that the binding rate of people with type A was the highest, and that of people with type O was the lowest. Our finding indicated that blood type A may be the biological marker for susceptibility to SARS-CoV-2 infection and may be associated with potential mediating of ACE2, but irrelevant to the clinical outcomes including ARDS, AKI, and death. These findings can provide new ideas for clinical diagnosis, treatment, and prevention of COVID-19

Gene-Centric Meta-Analysis of Lipid Traits in African, East Asian and Hispanic Populations

Meta-analyses of European populations has successfully identified genetic variants in over 100 loci associated with lipid levels, but our knowledge in other ethnicities remains limited. To address this, we performed dense genotyping of ∼2,000 candidate genes in 7,657 African Americans, 1,315 Hispanics and 841 East Asians, using the IBC array, a custom ∼50,000 SNP genotyping array. Meta-analyses confirmed 16 lipid loci previously established in European populations at genome-wide significance level, and found multiple independent association signals within these lipid loci. Initial discovery and in silico follow-up in 7,000 additional African American samples, confirmed two novel loci: rs5030359 within ICAM1 is associated with total cholesterol (TC) and low-density lipoprotein cholesterol (LDL-C) $(p = 8.8×10^{−7} and p = 1.5×10^{−6}$ respectively) and a nonsense mutation rs3211938 within CD36 is associated with high-density lipoprotein cholesterol (HDL-C) levels $(p = 13.5×10^{−12})$. The rs3211938-G allele, which is nearly absent in European and Asian populations, has been previously found to be associated with CD36 deficiency and shows a signature of selection in Africans and African Americans. Finally, we have evaluated the effect of SNPs established in European populations on lipid levels in multi-ethnic populations and show that most known lipid association signals span across ethnicities. However, differences between populations, especially differences in allele frequency, can be leveraged to identify novel signals, as shown by the discovery of ICAM1 and CD36 in the current report

Double Quantum Image Encryption Based on Arnold Transform and Qubit Random Rotation

Quantum image encryption offers major advantages over its classical counterpart in terms of key space, computational complexity, and so on. A novel double quantum image encryption approach based on quantum Arnold transform (QAT) and qubit random rotation is proposed in this paper, in which QAT is used to scramble pixel positions and the gray information is changed by utilizing random qubit rotation. Actually, the independent random qubit rotation operates once, respectively, in spatial and frequency domains with the help of quantum Fourier transform (QFT). The encryption process accomplishes pixel confusion and diffusion, and finally the noise-like cipher image is obtained. Numerical simulation and theoretical analysis verify that the method is valid and it shows superior performance in security and computational complexity

Energy Demodulation Algorithm for Flow Velocity Measurement of Oil-Gas-Water Three-Phase Flow

Flow velocity measurement was an important research of oil-gas-water three-phase flow parameter measurements. In order to satisfy the increasing demands for flow detection technology, the paper presented a gas-liquid phase flow velocity measurement method which was based on energy demodulation algorithm combing with time delay estimation technology. First, a gas-liquid phase separation method of oil-gas-water three-phase flow based on energy demodulation algorithm and blind signal separation technology was proposed. The separation of oil-gas-water three-phase signals which were sampled by conductance sensor performed well, so the gas-phase signal and the liquid-phase signal were obtained. Second, we used the time delay estimation technology to get the delay time of gas-phase signals and liquid-phase signals, respectively, and the gas-phase velocity and the liquid-phase velocity were derived. At last, the experiment was performed at oil-gas-water three-phase flow loop, and the results indicated that the measurement errors met the need of velocity measurement. So it provided a feasible method for gas-liquid phase velocity measurement of the oil-gas-water three-phase flow

Emulsions stabilized by nanofibers from bacterial cellulose: New potential food-grade Pickering emulsions

In the present work, we investigated the formation and stability of Pickering emulsions stabilized by nanoparticles generated from bacterial cellulose (BC) by hydrochloric acid hydrolysis. The resulting particles, called nanofibers, presented a ribbonlike shape with diameters of 30–80 nm and range in length from 100 nm to several micrometers. The obtained nanofibers showed good hydrophilic and lipophilic properties and had significant ability to reduce the surface tension of oil/water droplets from 48.55 ± 0.03 to 34.52 ± 0.05 mN/m. The oil-in-water Pickering emulsions with a peanut oil concentration of 15% (v/v) were stabilized by only 0.05% (w/v) nanofibers and displayed a narrow droplet size distribution and high intensity with an average droplet size of 15.00 ± 0.82 nm. The morphological studies confirmed the nano-scaled droplets of emulsions. The effects of pH values and temperatures on the creaming ability and physical stability were also evaluated by zeta-potential and droplet sizes. Results showed that emulsions displayed relatively lower creaming ability at pH < 7, while displayed optimal physical stability and dispersibility at pH ≥ 7. The temperature (20–100 °C) and time-dependent test (0–4 weeks) indicated that the Pickering emulsions stabilized by only 0.05% (w/v) nanofibers displayed excellent stability. Due to the sustainability and good bio-compatibility of nanofibers from BC, the developed emulsions stabilized by low concentration of nanofibers can be used as new food-grade Pickering emulsions and have great potential to deliver lipophilic bioactive substances in food industry

New Measurement Method of Oil-Water Two-Phase Flow with High Water Holdup and Low Rate by Phase State Regulation

Flow rate and holdup are two essential parameters to describe oil-water two-phase flow. The distribution of oil-water two-phase flow in the pipeline is very uneven, and there is a significant slippage between the phases. This makes it difficult to measure these two flow parameters. In this paper, a new measurement method of flow rate and holdup based on phase state regulation is proposed. The oil-water two-phase flow is adjusted to oil or water single-phase flow according to the time sequence by the phase state regulation, and the oil-water phase interface is measured with a conductance sensor. A wavelet transform based phase inflection point detection model is proposed to detect the oil-water phase change point. The experimental results show that the maximum measurement error of the flow rate of water is 3.73%, the maximum measurement error of the flow rate of oil is 3.68%, and the flow rate measurement repeatability is 0.0002. The accuracy of the measurement holdup is better than 3.23%, and the repeatability of the measurement holdup is 0.0003. The prototype designed based on this method has two advantages. One is that it is small in size, the other is that it does not depend on the accuracy of the sensor. Therefore, it can be widely used in oilfield ground measurement